1 Goal
The goal here is to access datasets from NCEI. These datasets include daily, nonthly and hourly datasets.
This particular script uses the rNOAA dataset that accesses the servers the National Center for Environmental Information (NCEI)
A link to the way you do this with the web system is here. We’ll be using R to do this for us.
2 Libraries
The following libraries are needed for this exercise.
Tidyverse Packages
tidyverse : Set of commonly-used Data Science packages for R that it can install and load all at once. In the long-run you probably also want to install the tidyverse package suite anyway. For this exercise this will include…
- ggplot2 : Create Elegant Data Visualizations Using the Grammar of Graphics
- tibble : Simple Data Frames
- tidyr : Tools for shepherding data in data frames.
- readr : Read Rectangular Text Data
- purr : Functional Programming Tools
- dplyr : A grammar of data manipulation
- stringr : Simple, Consistent Wrappers for Common String Operations
- forcats : Tools for Working with Categorical Variables (Factors)
lubridate : Time and Date Management
ggridges : Ridgeline plots with ggplot2
Other Packages
- rnoaa : Climate Data Online Services from NCEI
# Libraries
# Tidyverse resources
library(package = "tidyverse") # Multiple Tidyverse Resources
library(package = "lubridate") # Date-Time Control
Attaching package: ‘lubridate’
The following object is masked from ‘package:base’:
date
library(package = "ggridges") # Ridgeline plots with ggplot2
# NOAA Libraries
library(package = "rnoaa") # NCEI Data Retrieval Package
Registered S3 method overwritten by 'hoardr':
method from
print.cache_info httr
3 Searching for Data
3.1. Available Datasets
The hardest part of this took is to look for the data. For our example there are 11 datasets to access using this tool. (rnoaa will let you get other data like storm and tornado data but we’re not doing that yet.) They are listed here using the ncdc_datasets() command.
# Use NCDC Datasets to gret available datasets
ncdc_datasets()$data
NA
NA
To access these fields you can use the “id” value in the table above. So for daily summaries from the Global Historical Climate Data Network (GHCN) you would use “GHCND”.
3.2. Available Stations
Once you have dataset the fun begins with being able to find a given station or set of stations.
There are a couple ways to do this and they use ncdc_stations()
You can search by city, county, state, country, USGS watershed HUC number, zip code..
The different search methods are:
ncdc_locs_cats()$data
NA
The best way to search for a region is to use the web interface from NCEI/NCDC and get a “Location ID”
https://www.ncdc.noaa.gov/cdo-web/search
Use the “search for” and you should be able to pull from one of the above categories in the lower pull-down menu to get the code.
Set the limit to 1000 to make sure that you can get the total number of available of stations (so long as they are … othwewise you will get only 25 which is the default for the function)
For example:
3.2.1 Searching based on a city location
Stations within a somewhat arbitrary radius of the “CITY” of Sydney, Australia, can be found with “locationid” CITY:AS000010
# Station List for Sydney, Australia
ncdc_ids = ncdc_stations(locationid = 'CITY:AS000010',
datasetid = 'GHCND',
limit = 1000)
ncdc_ids$data
NA
3.2.2 Searching based on a stations inside a given US County
Stations in Pennington “CNTY” can be found using “locatoinid” FIPS:46103 (sorry you can’t tunnel down to subdivisions in other countries like New South Wales, AU)
# Station List for Pennington County, SD
ncdc_ids = ncdc_stations(locationid = 'FIPS:46103',
datasetid = 'GHCND',
limit = 1000)
ncdc_ids$data
NA
3.2.3 Searching based on a stations inside a given US State
Stations in the “ST” of South Dakota, FIPS:46
# Station List for South Dakota
ncdc_ids = ncdc_stations(locationid = 'FIPS:46',
datasetid = 'GHCND',
limit = 1000)
ncdc_ids$data
NA
3.2.4 Searching based on a stations inside a given River Basin
Stations in the Rapid Creek Basin or other named USGS Stream Unit search for the “Hydrologic Accounting Unit” (HYD_ACC), HUC:10120110 (this only works for those basins in the US)’
(That station maked Lead 11S, or "11 km south of Lead is in the Rapid City Basin, not inside of Lead.)
# Station List for the Rapid Creek River Basin
ncdc_ids = ncdc_stations(locationid = 'HUC:10120110',
datasetid = 'GHCND',
limit = 1000)
ncdc_ids$data
NA
3.2.5 Searching based on a stations inside a given country
Stations in North Korea that they allow us to see, FIPS:NK
(those two last stations that are in “South Korea” are radar stations in South Korea whose coverages cross the border into North Korea)
# Station List for the North Korea
ncdc_ids = ncdc_stations(locationid = 'FIPS:KN',
limit = 1000)
ncdc_ids$data
NA
4 Pulling In Data and Getting It Into Shape.
So let’s now try getting data from the Rapid City Airport.
Rapid City Airport (using the data from above for Pennington County, the GHCN station ID for the airport is… GHCND:USW00024090)
We can get the details for the station using the option “stationid = ‘GHCND:USW00024090’”
# Station Details for Rapid City Airport , SD
stationid_for_ncdcstations = 'GHCND:USW00024090'
stationid_for_ghcn_pull = 'USW00024090'
# Station Details for Corvalis Or (OSU)
stationid_for_ncdcstations = 'GHCND:USC00351862'
stationid_for_ghcn_pull = 'USC00351862'
# Station Details for Cheatah Reserve
stationid_for_ncdcstations = 'GHCND:WA009181280'
stationid_for_ghcn_pull = 'WA009181280'
# Station Details for NRMCAS
stationid_for_ncdcstations = 'GHCND:USW00093727'
stationid_for_ghcn_pull = 'USW00093727'
ncdc_ids = ncdc_stations(stationid = stationid_for_ncdcstations)
ncdc_ids = ncdc_ids$data
Also to get an inventory of the avaialble data for this station we can use the ncdc_datatypes() command.
# Get Available Parameters for a given station from a specific dataset
ncdc_datatypes(datasetid = 'GHCND',
stationid = stationid_for_ncdcstations)$data
NA
4.1 Pulling the Raw Data from NCDC/NCEI
To pull the daily GHCN data for this station we use the meteo_tidy_ghcnd() function. We should be just able to ask, again, for the station ID for the airport.
# Pull the Raw Climate Data from a Single Station
ghcn_data = meteo_tidy_ghcnd(stationid = stationid_for_ghcn_pull,
keep_flags = TRUE,
var = "all",
date_min = NULL,
date_max = NULL)
file path: /Users/wjc/Library/Caches/rnoaa/ghcnd/USW00093727.dly
file last updated: 2020-01-27 10:35:48
file min/max dates: 1955-01-01 / 2020-01-31
# drop empty variable columns (they should already have been dropped though)
ghcn_data = ghcn_data[,colSums(is.na(ghcn_data))<nrow(ghcn_data)]
ghcn_data
NA
The downside in the above output is that there are no units mentioned here. But they are available on the original data website here:
https://www1.ncdc.noaa.gov/pub/data/ghcn/daily/readme.txt
From the dataset the five basic parameters are
| PRCP |
Precipitation wrt 0000-0000 Local Time |
⅒ of mm |
| SNOW |
Snowfall wrt 0000-0000 Local Time |
mm |
| SNWD |
Snow depth wrt 0000-0000 Local Time |
mm |
| TMAX |
Maximum temperature wrt 0000-0000 Local Time |
⅒ of °C |
| TMIN |
Minimum temperature wrt 0000-0000 Local Time |
⅒ of °C |
And you may have
| TAVG |
Average temperature wrt 0000-0000 UTC |
⅒ of °C |
These really are the only parameters I normally pull.
I’m doing this by brute force and changing the temperature units to °F while I am at it…
# Changing Units
ghcn_data$tmax = ghcn_data$tmax/10 * 9. / 5. + 32
ghcn_data$tmin = ghcn_data$tmin/10 * 9. / 5. + 32
4.3. Patching any Missing Values
Here, we will do a check to ensure a continous dataset that places “NA” for missing records left_join() function
# create a date frame to set us up to accomodate missing data
daily_time_frame = tibble(date = seq.Date(from = min(ghcn_data$date),
to = max(ghcn_data$date),
by = "1 day"))
ghcn_data = left_join(daily_time_frame,ghcn_data, by="date")
print(ghcn_data)
remove(daily_time_frame)
And with that we can now [finally] play!
5 Make a Simple Plot
Let’s make a simple plot for Tmin
ggplot(data = ghcn_data) + # use ghcn_data as the source of the data
aes(x = date, # select specific fields to plot
y = tmin) +
theme_bw() + # use a very simple ploting theme
ggtitle(label = "Daily Global Historical Climate Data",
subtitle = str_c(ncdc_ids$name,
" :: ",
stationid_for_ncdcstations)) +
xlab(label = "Date") +
ylab(label = "Minimum Daily Temperature (°F)") +
geom_line(color = "blue") # make a simple line plot (and make it pink)

NA
NA
We can alsos take a close look at any flags for that parameter.
# get the tmin record in this case.
ghcn_tmin = ghcn_data %>% select(c("date",
ends_with("tmin")))
ghcn_tmin %>% filter(qflag_tmin != " ")
NA
ggplot(data = ghcn_data) + # use ghcn_data as the source of the data
aes(x = date, # select specific fields to plot
y = tmax) +
theme_bw() + # use a very simple ploting theme
ggtitle(label = "Daily Global Historical Climate Data",
subtitle = str_c(ncdc_ids$name,
" :: ",
stationid_for_ncdcstations)) +
xlab(label = "Date") +
ylab(label = "Maximum Daily Temperature (°F)") +
geom_line(color = "red") # make a simple line plot (and make it pink)

NA
NA
We can alsos take a close look at any flags for that parameter.
# get the tmin record in this case.
ghcn_tmax = ghcn_data %>% select(c("date",
ends_with("tmax")))
ghcn_tmax %>% filter(qflag_tmax != " ")
NA
6 Make a Fancy Plot to Show Changes in Climate
Now let’s fancy. Let’s see how max and min tempertures for each month changes over time.
Let’s chose some 30-y periods at the beginning and end of the record just as a demo. (You can add more periods if you want)
Let’s chose two parameters. Max and Minimum Temperatures
First, let’s change the data frame so that we have only min and max temperatures
# pull only the min and max temps
temp_data = ghcn_data %>% select(c(date,
tmax,
tmin))
We’ll start by creating an additional data column to represent the decade we’ll use the year() function to do this.
# Create a year parameter
temp_data$year = year(temp_data$date)
# Turn it into a string that displays the start and end year of the decade
And add another variable for the calendar month
# add month
temp_data$month = month(x = temp_data$date,
label = TRUE,
abbr = TRUE)
And filter only the periods we want to use
# first full year (use the lowest date where the DOY via yday == 1)
period_1_year_start = year(min(temp_data$date[yday(temp_data$date)==1]))
period_1_year_end = period_1_year_start + (30 - 1)
period_1_string = str_c(period_1_year_start,
"-",
period_1_year_end,
sep = "")
# most recent year period
period_3_year_start = 2018 - (30 - 1)
period_3_year_end = 2018
period_3_string = str_c(period_3_year_start,
"-",
period_3_year_end,
sep = "")
# a period in the middle
period_2_year_start = round((period_1_year_start + period_3_year_start) / 2 )
period_2_year_end = round((period_1_year_end + period_3_year_end) / 2 )
period_2_string = str_c(period_2_year_start,
"-",
period_2_year_end,
sep = "")
# extract these two periods
temp_data = temp_data %>%
filter(( (year(date) >= period_1_year_start) &
(year(date) <= period_1_year_end ) ) |
( (year(date) >= period_2_year_start) &
(year(date) <= period_2_year_end ) ) |
( (year(date) >= period_3_year_start) &
(year(date) <= period_3_year_end ) ) )
temp_data = temp_data %>%
mutate(Period = case_when((year(date) >= period_1_year_start) & (year(date) <= period_1_year_end ) ~ period_1_string,
(year(date) >= period_2_year_start) & (year(date) <= period_2_year_end ) ~ period_2_string,
(year(date) >= period_3_year_start) & (year(date) <= period_3_year_end ) ~ period_3_string))
temp_data$Period = as.factor(temp_data$Period)
Make a ridge plot by month
# ridgeplot
ggplot(data = temp_data) +
aes(x = tmax,
y = fct_rev(month),
fill = Period) +
theme_bw() +
ggtitle(label = "Daily Global Historical Climate Data",
subtitle = str_c(ncdc_ids$name,
" :: ",
stationid_for_ncdcstations)) +
ylab(label = "Month") +
xlab(label = "Daily Temperature (°F)") +
scale_fill_manual(values=c("blue",
"yellow",
"red"))+
geom_density_ridges2(alpha = 0.25,
color = NA) +
geom_density_ridges2(mapping = aes(x = tmin,
y = fct_rev(month),
fill = Period),
alpha = 0.25,
color = NA)

SCAVENGER HUNT
- Take the gchn data set
- Find Yearly Max-Max Temp, Min-Min Temp, Max-Daily Precip
- Try your method for getting max min returns
write.csv(x = ghcn_data,
file = str_c("./GHCN_",
unique(stationid_for_ghcn_pull),
".csv"))
LS0tCnRpdGxlOiAiSW1wb3J0aW5nIERhaWx5IFN1bW1hcnkgRGF0YSBmcm9tIHRoZSBOYXRpb25hbCBDbGltYXRpYyBEYXRhIENlbnRlciIKb3V0cHV0OiBodG1sX25vdGVib29rCi0tLQoKCiMgMSBHb2FsCgpUaGUgZ29hbCBoZXJlIGlzIHRvIGFjY2VzcyBkYXRhc2V0cyBmcm9tIE5DRUkuICBUaGVzZSBkYXRhc2V0cyBpbmNsdWRlIGRhaWx5LCBub250aGx5IGFuZCBob3VybHkgZGF0YXNldHMuICAKClRoaXMgcGFydGljdWxhciBzY3JpcHQgdXNlcyB0aGUgck5PQUEgZGF0YXNldCB0aGF0IGFjY2Vzc2VzIHRoZSBzZXJ2ZXJzIHRoZSBOYXRpb25hbCBDZW50ZXIgZm9yIEVudmlyb25tZW50YWwgSW5mb3JtYXRpb24gKE5DRUkpCgpBIGxpbmsgdG8gdGhlIHdheSB5b3UgZG8gdGhpcyB3aXRoIHRoZSB3ZWIgc3lzdGVtIGlzIFtoZXJlXShodHRwczovL3d3dy5uY2RjLm5vYWEuZ292L2Nkby13ZWIvd2Vic2VydmljZXMvdjIpLiAgV2UnbGwgYmUgdXNpbmcgUiB0byBkbyB0aGlzIGZvciB1cy4KCiMgMiAgTGlicmFyaWVzCgpUaGUgZm9sbG93aW5nIGxpYnJhcmllcyBhcmUgbmVlZGVkIGZvciB0aGlzIGV4ZXJjaXNlLgoKVGlkeXZlcnNlIFBhY2thZ2VzCgogICsgW3RpZHl2ZXJzZV0oaHR0cHM6Ly93d3cudGlkeXZlcnNlLm9yZykgOiBTZXQgb2YgY29tbW9ubHktdXNlZCBEYXRhIFNjaWVuY2UgcGFja2FnZXMgZm9yIFIgdGhhdCBpdCBjYW4gaW5zdGFsbCBhbmQgbG9hZCBhbGwgYXQgb25jZS4gSW4gdGhlIGxvbmctcnVuIHlvdSBwcm9iYWJseSBhbHNvIHdhbnQgdG8gaW5zdGFsbCB0aGUgdGlkeXZlcnNlIHBhY2thZ2Ugc3VpdGUgYW55d2F5LiBGb3IgdGhpcyBleGVyY2lzZSB0aGlzIHdpbGwgaW5jbHVkZS4uLiAgIAogICAgLSBbZ2dwbG90Ml0oaHR0cHM6Ly9nZ3Bsb3QyLnRpZHl2ZXJzZS5vcmcpIDogQ3JlYXRlIEVsZWdhbnQgRGF0YSBWaXN1YWxpemF0aW9ucyBVc2luZyB0aGUgR3JhbW1hciBvZiBHcmFwaGljcwogICAgLSBbdGliYmxlXShodHRwczovL3RpYmJsZS50aWR5dmVyc2Uub3JnKSA6IFNpbXBsZSBEYXRhIEZyYW1lcwogICAgLSBbdGlkeXJdKGh0dHBzOi8vdGlkeXIudGlkeXZlcnNlLm9yZykgOiBUb29scyBmb3Igc2hlcGhlcmRpbmcgZGF0YSBpbiBkYXRhIGZyYW1lcy4KICAgIC0gW3JlYWRyXShodHRwczovL3JlYWRyLnRpZHl2ZXJzZS5vcmcpIDogUmVhZCBSZWN0YW5ndWxhciBUZXh0IERhdGEKICAgIC0gW3B1cnJdKGh0dHBzOi8vcHVycnIudGlkeXZlcnNlLm9yZykgOiBGdW5jdGlvbmFsIFByb2dyYW1taW5nIFRvb2xzCiAgICAtIFtkcGx5cl0oaHR0cHM6Ly9kcGx5ci50aWR5dmVyc2Uub3JnKSA6IEEgZ3JhbW1hciBvZiBkYXRhIG1hbmlwdWxhdGlvbgogICAgLSBbc3RyaW5ncl0oaHR0cHM6Ly9zdHJpbmdyLnRpZHl2ZXJzZS5vcmcpIDogU2ltcGxlLCBDb25zaXN0ZW50IFdyYXBwZXJzIGZvciBDb21tb24gU3RyaW5nIE9wZXJhdGlvbnMKICAgIC0gW2ZvcmNhdHNdKGh0dHBzOi8vZm9yY2F0cy50aWR5dmVyc2Uub3JnKSA6IFRvb2xzIGZvciBXb3JraW5nIHdpdGggQ2F0ZWdvcmljYWwgVmFyaWFibGVzIChGYWN0b3JzKQogICAgCiAgKyBbbHVicmlkYXRlXShodHRwczovL2x1YnJpZGF0ZS50aWR5dmVyc2Uub3JnKSA6IFRpbWUgYW5kIERhdGUgTWFuYWdlbWVudAogIAogICsgW2dncmlkZ2VzXShodHRwczovL3d3dy5yZG9jdW1lbnRhdGlvbi5vcmcvcGFja2FnZXMvZ2dyaWRnZXMvdmVyc2lvbnMvMC41LjEpIDogUmlkZ2VsaW5lIHBsb3RzIHdpdGggZ2dwbG90MgogIApPdGhlciBQYWNrYWdlcwoKICArIFtybm9hYV0oaHR0cHM6Ly93d3cucmRvY3VtZW50YXRpb24ub3JnL3BhY2thZ2VzL3Jub2FhL3ZlcnNpb25zLzAuMi4wKSA6IENsaW1hdGUgRGF0YSBPbmxpbmUgU2VydmljZXMgZnJvbSBOQ0VJIAogICAgCmBgYHtyfQoKIyBMaWJyYXJpZXMKCiAgIyBUaWR5dmVyc2UgcmVzb3VyY2VzCgogIGxpYnJhcnkocGFja2FnZSA9ICJ0aWR5dmVyc2UiKSAjIE11bHRpcGxlIFRpZHl2ZXJzZSBSZXNvdXJjZXMKICBsaWJyYXJ5KHBhY2thZ2UgPSAibHVicmlkYXRlIikgIyBEYXRlLVRpbWUgQ29udHJvbAogIGxpYnJhcnkocGFja2FnZSA9ICJnZ3JpZGdlcyIpICAjIFJpZGdlbGluZSBwbG90cyB3aXRoIGdncGxvdDIKCgogICMgTk9BQSBMaWJyYXJpZXMKCiAgbGlicmFyeShwYWNrYWdlID0gInJub2FhIikgIyBOQ0VJICBEYXRhIFJldHJpZXZhbCBQYWNrYWdlCgpgYGAKCiMgMyBTZWFyY2hpbmcgZm9yIERhdGEKCiMjIDMuMS4gQXZhaWxhYmxlIERhdGFzZXRzCgpUaGUgaGFyZGVzdCBwYXJ0IG9mIHRoaXMgdG9vayBpcyB0byBsb29rIGZvciB0aGUgZGF0YS4gIEZvciBvdXIgZXhhbXBsZSB0aGVyZSBhcmUgMTEgZGF0YXNldHMgdG8gYWNjZXNzIHVzaW5nIHRoaXMgdG9vbC4gKHJub2FhIHdpbGwgbGV0IHlvdSBnZXQgb3RoZXIgZGF0YSBsaWtlIHN0b3JtIGFuZCB0b3JuYWRvIGRhdGEgYnV0IHdlJ3JlIG5vdCBkb2luZyB0aGF0IHlldC4pIFRoZXkgYXJlIGxpc3RlZCBoZXJlIHVzaW5nIHRoZSBbbmNkY19kYXRhc2V0cygpXShodHRwczovL3d3dy5yZG9jdW1lbnRhdGlvbi5vcmcvcGFja2FnZXMvcm5vYWEvdmVyc2lvbnMvMC44LjQvdG9waWNzL25jZGNfZGF0YXNldHMpIGNvbW1hbmQuCgpgYGB7cn0KCiMgVXNlIE5DREMgRGF0YXNldHMgdG8gZ3JldCBhdmFpbGFibGUgZGF0YXNldHMKCiAgbmNkY19kYXRhc2V0cygpJGRhdGEKICAKCmBgYAoKVG8gYWNjZXNzIHRoZXNlIGZpZWxkcyB5b3UgY2FuIHVzZSB0aGUgImlkIiB2YWx1ZSBpbiB0aGUgdGFibGUgYWJvdmUuICBTbyBmb3IgZGFpbHkgc3VtbWFyaWVzIGZyb20gdGhlIEdsb2JhbCBIaXN0b3JpY2FsIENsaW1hdGUgRGF0YSBOZXR3b3JrIChHSENOKSB5b3Ugd291bGQgdXNlICJHSENORCIuCgojIyAzLjIuICBBdmFpbGFibGUgU3RhdGlvbnMKCk9uY2UgeW91IGhhdmUgZGF0YXNldCB0aGUgZnVuIGJlZ2lucyB3aXRoIGJlaW5nIGFibGUgdG8gZmluZCBhIGdpdmVuIHN0YXRpb24gb3Igc2V0IG9mIHN0YXRpb25zLiAgCgpUaGVyZSBhcmUgYSBjb3VwbGUgd2F5cyB0byBkbyB0aGlzIGFuZCB0aGV5IHVzZSBbbmNkY19zdGF0aW9ucygpXShodHRwczovL3d3dy5yZG9jdW1lbnRhdGlvbi5vcmcvcGFja2FnZXMvcm5vYWEvdmVyc2lvbnMvMC44LjQvdG9waWNzL25jZGNfc3RhdGlvbnMpCgpZb3UgY2FuIHNlYXJjaCBieSBjaXR5LCBjb3VudHksIHN0YXRlLCBjb3VudHJ5LCBVU0dTIHdhdGVyc2hlZCBIVUMgbnVtYmVyLCB6aXAgY29kZS4uCgpUaGUgZGlmZmVyZW50IHNlYXJjaCBtZXRob2RzIGFyZToKCmBgYHtyfQoKbmNkY19sb2NzX2NhdHMoKSRkYXRhCgpgYGAKClRoZSBiZXN0IHdheSB0byBzZWFyY2ggZm9yIGEgcmVnaW9uIGlzIHRvIHVzZSB0aGUgd2ViIGludGVyZmFjZSBmcm9tIE5DRUkvTkNEQyBhbmQgZ2V0IGEgIkxvY2F0aW9uIElEIgoKW2h0dHBzOi8vd3d3Lm5jZGMubm9hYS5nb3YvY2RvLXdlYi9zZWFyY2hdKGh0dHBzOi8vd3d3Lm5jZGMubm9hYS5nb3YvY2RvLXdlYi9zZWFyY2gpCgpVc2UgdGhlICJzZWFyY2ggZm9yIiBhbmQgeW91IHNob3VsZCBiZSBhYmxlIHRvIHB1bGwgZnJvbSBvbmUgb2YgdGhlIGFib3ZlIGNhdGVnb3JpZXMgaW4gdGhlIGxvd2VyIHB1bGwtZG93biBtZW51IHRvIGdldCB0aGUgY29kZS4KCgoKU2V0IHRoZSBsaW1pdCB0byAxMDAwIHRvIG1ha2Ugc3VyZSB0aGF0IHlvdSBjYW4gZ2V0IHRoZSB0b3RhbCBudW1iZXIgb2YgYXZhaWxhYmxlIG9mIHN0YXRpb25zIChzbyBsb25nIGFzIHRoZXkgYXJlIC4uLiBvdGh3ZXdpc2UgeW91IHdpbGwgZ2V0IG9ubHkgMjUgd2hpY2ggaXMgdGhlIGRlZmF1bHQgZm9yIHRoZSBmdW5jdGlvbikKCkZvciBleGFtcGxlOgoKIyMjIDMuMi4xIFNlYXJjaGluZyBiYXNlZCBvbiBhIGNpdHkgbG9jYXRpb24KClN0YXRpb25zIHdpdGhpbiBhIHNvbWV3aGF0IGFyYml0cmFyeSByYWRpdXMgb2YgdGhlICJDSVRZIiBvZiBTeWRuZXksIEF1c3RyYWxpYSwgY2FuIGJlIGZvdW5kIHdpdGggImxvY2F0aW9uaWQiICoqQ0lUWTpBUzAwMDAxMCoqCgpgYGB7cn0KCiMgU3RhdGlvbiBMaXN0IGZvciBTeWRuZXksIEF1c3RyYWxpYQoKbmNkY19pZHMgPSBuY2RjX3N0YXRpb25zKGxvY2F0aW9uaWQgPSAnQ0lUWTpBUzAwMDAxMCcsIAogICAgICAgICAgICAgICAgICAgICAgICAgZGF0YXNldGlkICA9ICdHSENORCcsCiAgICAgICAgICAgICAgICAgICAgICAgICBsaW1pdCAgICAgID0gMTAwMCkKCm5jZGNfaWRzJGRhdGEKCmBgYAoKIyMjIDMuMi4yIFNlYXJjaGluZyBiYXNlZCBvbiBhIHN0YXRpb25zIGluc2lkZSBhIGdpdmVuIFVTIENvdW50eQoKU3RhdGlvbnMgaW4gUGVubmluZ3RvbiAiQ05UWSIgY2FuIGJlIGZvdW5kIHVzaW5nICJsb2NhdG9pbmlkIiAqKkZJUFM6NDYxMDMqKiAoc29ycnkgeW91IGNhbid0IHR1bm5lbCBkb3duIHRvIHN1YmRpdmlzaW9ucyBpbiBvdGhlciBjb3VudHJpZXMgbGlrZSBOZXcgU291dGggV2FsZXMsIEFVKQoKYGBge3J9CgojIFN0YXRpb24gTGlzdCBmb3IgUGVubmluZ3RvbiBDb3VudHksIFNECgpuY2RjX2lkcyA9IG5jZGNfc3RhdGlvbnMobG9jYXRpb25pZCA9ICdGSVBTOjQ2MTAzJywgCiAgICAgICAgICAgICAgICAgICAgICAgICBkYXRhc2V0aWQgID0gJ0dIQ05EJywKICAgICAgICAgICAgICAgICAgICAgICAgIGxpbWl0ICAgICAgPSAxMDAwKQoKbmNkY19pZHMkZGF0YQoKYGBgCgojIyMgMy4yLjMgU2VhcmNoaW5nIGJhc2VkIG9uIGEgc3RhdGlvbnMgaW5zaWRlIGEgZ2l2ZW4gVVMgU3RhdGUKClN0YXRpb25zIGluIHRoZSAiU1QiIG9mIFNvdXRoIERha290YSwgKipGSVBTOjQ2KiogIAoKYGBge3J9CgojIFN0YXRpb24gTGlzdCBmb3IgU291dGggRGFrb3RhCgpuY2RjX2lkcyA9IG5jZGNfc3RhdGlvbnMobG9jYXRpb25pZCA9ICdGSVBTOjQ2JywgCiAgICAgICAgICAgICAgICAgICAgICAgICBkYXRhc2V0aWQgID0gJ0dIQ05EJywKICAgICAgICAgICAgICAgICAgICAgICAgIGxpbWl0ICAgICAgPSAxMDAwKQoKbmNkY19pZHMkZGF0YQoKYGBgCgojIyMgMy4yLjQgU2VhcmNoaW5nIGJhc2VkIG9uIGEgc3RhdGlvbnMgaW5zaWRlIGEgZ2l2ZW4gUml2ZXIgQmFzaW4KClN0YXRpb25zIGluIHRoZSBSYXBpZCBDcmVlayBCYXNpbiBvciBvdGhlciBuYW1lZCBVU0dTIFN0cmVhbSBVbml0IHNlYXJjaCBmb3IgdGhlICJIeWRyb2xvZ2ljIEFjY291bnRpbmcgVW5pdCIgKEhZRF9BQ0MpLCAqKkhVQzoxMDEyMDExMCoqICh0aGlzIG9ubHkgd29ya3MgZm9yIHRob3NlIGJhc2lucyBpbiB0aGUgVVMpJwoKKFRoYXQgc3RhdGlvbiBtYWtlZCBMZWFkIDExUywgb3IgIjExIGttIHNvdXRoIG9mIExlYWQgaXMgaW4gdGhlIFJhcGlkIENpdHkgQmFzaW4sIG5vdCBpbnNpZGUgb2YgTGVhZC4pCgpgYGB7cn0KCiMgU3RhdGlvbiBMaXN0IGZvciB0aGUgUmFwaWQgQ3JlZWsgUml2ZXIgQmFzaW4KCm5jZGNfaWRzID0gbmNkY19zdGF0aW9ucyhsb2NhdGlvbmlkID0gJ0hVQzoxMDEyMDExMCcsIAogICAgICAgICAgICAgICAgICAgICAgICAgZGF0YXNldGlkICA9ICdHSENORCcsCiAgICAgICAgICAgICAgICAgICAgICAgICBsaW1pdCAgICAgID0gMTAwMCkKCm5jZGNfaWRzJGRhdGEKCmBgYAoKIyMjIDMuMi41IFNlYXJjaGluZyBiYXNlZCBvbiBhIHN0YXRpb25zIGluc2lkZSBhIGdpdmVuIGNvdW50cnkKClN0YXRpb25zIGluIE5vcnRoIEtvcmVhIHRoYXQgdGhleSBhbGxvdyB1cyB0byBzZWUsICoqRklQUzpOSyoqICAKCih0aG9zZSB0d28gbGFzdCBzdGF0aW9ucyB0aGF0IGFyZSBpbiAiU291dGggS29yZWEiIGFyZSByYWRhciBzdGF0aW9ucyBpbiBTb3V0aCBLb3JlYSB3aG9zZSBjb3ZlcmFnZXMgY3Jvc3MgdGhlIGJvcmRlciBpbnRvIE5vcnRoIEtvcmVhKQoKYGBge3J9CgojIFN0YXRpb24gTGlzdCBmb3IgdGhlIE5vcnRoIEtvcmVhCgpuY2RjX2lkcyA9IG5jZGNfc3RhdGlvbnMobG9jYXRpb25pZCA9ICdGSVBTOktOJywgCiAgICAgICAgICAgICAgICAgICAgICAgICBsaW1pdCAgICAgID0gMTAwMCkKCm5jZGNfaWRzJGRhdGEKCmBgYAoKCiMgNCBQdWxsaW5nIEluIERhdGEgYW5kIEdldHRpbmcgSXQgSW50byBTaGFwZS4KClNvIGxldCdzIG5vdyB0cnkgZ2V0dGluZyBkYXRhIGZyb20gdGhlIFJhcGlkIENpdHkgQWlycG9ydC4KClJhcGlkIENpdHkgQWlycG9ydCAodXNpbmcgdGhlIGRhdGEgZnJvbSBhYm92ZSBmb3IgUGVubmluZ3RvbiBDb3VudHksIHRoZSBHSENOIHN0YXRpb24gSUQgZm9yIHRoZSBhaXJwb3J0IGlzLi4uICoqR0hDTkQ6VVNXMDAwMjQwOTAqKikKCldlIGNhbiBnZXQgdGhlIGRldGFpbHMgZm9yIHRoZSBzdGF0aW9uIHVzaW5nIHRoZSBvcHRpb24gInN0YXRpb25pZCA9ICdHSENORDpVU1cwMDAyNDA5MCciCgpgYGB7cn0KCiMgU3RhdGlvbiBEZXRhaWxzIGZvciBSYXBpZCBDaXR5IEFpcnBvcnQgLCBTRAoKc3RhdGlvbmlkX2Zvcl9uY2Rjc3RhdGlvbnMgPSAnR0hDTkQ6VVNXMDAwMjQwOTAnCgpzdGF0aW9uaWRfZm9yX2doY25fcHVsbCAgICA9ICdVU1cwMDAyNDA5MCcKCiMgU3RhdGlvbiBEZXRhaWxzIGZvciBDb3J2YWxpcyBPciAoT1NVKQoKCnN0YXRpb25pZF9mb3JfbmNkY3N0YXRpb25zID0gJ0dIQ05EOlVTQzAwMzUxODYyJwpzdGF0aW9uaWRfZm9yX2doY25fcHVsbCAgICA9ICdVU0MwMDM1MTg2MicKCgojIFN0YXRpb24gRGV0YWlscyBmb3IgQ2hlYXRhaCBSZXNlcnZlCgpzdGF0aW9uaWRfZm9yX25jZGNzdGF0aW9ucyA9ICdHSENORDpXQTAwOTE4MTI4MCcKc3RhdGlvbmlkX2Zvcl9naGNuX3B1bGwgICAgPSAnV0EwMDkxODEyODAnCgoKCgojIFN0YXRpb24gRGV0YWlscyBmb3IgTlJNQ0FTCgpzdGF0aW9uaWRfZm9yX25jZGNzdGF0aW9ucyA9ICdHSENORDpVU1cwMDA5MzcyNycKc3RhdGlvbmlkX2Zvcl9naGNuX3B1bGwgICAgPSAnVVNXMDAwOTM3MjcnCgpuY2RjX2lkcyA9IG5jZGNfc3RhdGlvbnMoc3RhdGlvbmlkID0gc3RhdGlvbmlkX2Zvcl9uY2Rjc3RhdGlvbnMpCgpuY2RjX2lkcyA9IG5jZGNfaWRzJGRhdGEKCmBgYAoKCgpBbHNvIHRvIGdldCBhbiBpbnZlbnRvcnkgb2YgdGhlIGF2YWlhbGJsZSBkYXRhIGZvciB0aGlzIHN0YXRpb24gd2UgY2FuIHVzZSB0aGUgW25jZGNfZGF0YXR5cGVzKCldKGh0dHBzOi8vd3d3LnJkb2N1bWVudGF0aW9uLm9yZy9wYWNrYWdlcy9ybm9hYS92ZXJzaW9ucy8wLjguNC90b3BpY3MvbmNkY19kYXRhdHlwZXMpIGNvbW1hbmQuCgpgYGB7cn0KCiMgR2V0IEF2YWlsYWJsZSBQYXJhbWV0ZXJzIGZvciBhIGdpdmVuIHN0YXRpb24gZnJvbSBhIHNwZWNpZmljIGRhdGFzZXQKCm5jZGNfZGF0YXR5cGVzKGRhdGFzZXRpZCA9ICdHSENORCcsCiAgICAgICAgICAgICAgIHN0YXRpb25pZCA9IHN0YXRpb25pZF9mb3JfbmNkY3N0YXRpb25zKSRkYXRhCgpgYGAKCiMjIDQuMSBQdWxsaW5nIHRoZSBSYXcgRGF0YSBmcm9tIE5DREMvTkNFSQoKVG8gcHVsbCB0aGUgZGFpbHkgR0hDTiBkYXRhIGZvciB0aGlzIHN0YXRpb24gd2UgdXNlIHRoZSBbbWV0ZW9fdGlkeV9naGNuZCgpXShodHRwczovL3d3dy5yZG9jdW1lbnRhdGlvbi5vcmcvcGFja2FnZXMvcm5vYWEvdmVyc2lvbnMvMC44LjQvdG9waWNzL21ldGVvX3RpZHlfZ2hjbmQpIGZ1bmN0aW9uLiAgV2Ugc2hvdWxkIGJlIGp1c3QgYWJsZSB0byBhc2ssIGFnYWluLCBmb3IgdGhlIHN0YXRpb24gSUQgZm9yIHRoZSBhaXJwb3J0LgoKYGBge3J9CgojIFB1bGwgdGhlIFJhdyBDbGltYXRlIERhdGEgZnJvbSBhIFNpbmdsZSBTdGF0aW9uCgpnaGNuX2RhdGEgPSBtZXRlb190aWR5X2doY25kKHN0YXRpb25pZCAgPSBzdGF0aW9uaWRfZm9yX2doY25fcHVsbCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICBrZWVwX2ZsYWdzID0gVFJVRSwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdmFyICAgICAgICA9ICJhbGwiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgIGRhdGVfbWluICAgPSBOVUxMLCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICBkYXRlX21heCAgID0gTlVMTCkKCiMgZHJvcCBlbXB0eSB2YXJpYWJsZSBjb2x1bW5zICh0aGV5IHNob3VsZCBhbHJlYWR5IGhhdmUgYmVlbiBkcm9wcGVkIHRob3VnaCkKCmdoY25fZGF0YSA9IGdoY25fZGF0YVssY29sU3Vtcyhpcy5uYShnaGNuX2RhdGEpKTxucm93KGdoY25fZGF0YSldCgpnaGNuX2RhdGEKCmBgYAoKCgpUaGUgZG93bnNpZGUgaW4gdGhlIGFib3ZlIG91dHB1dCBpcyB0aGF0IHRoZXJlIGFyZSBubyB1bml0cyBtZW50aW9uZWQgaGVyZS4gIEJ1dCB0aGV5ICphcmUqIGF2YWlsYWJsZSBvbiB0aGUgb3JpZ2luYWwgZGF0YSB3ZWJzaXRlIGhlcmU6CgpbaHR0cHM6Ly93d3cxLm5jZGMubm9hYS5nb3YvcHViL2RhdGEvZ2hjbi9kYWlseS9yZWFkbWUudHh0XShodHRwczovL3d3dzEubmNkYy5ub2FhLmdvdi9wdWIvZGF0YS9naGNuL2RhaWx5L3JlYWRtZS50eHQpCgpGcm9tIHRoZSBkYXRhc2V0IHRoZSBmaXZlIGJhc2ljIHBhcmFtZXRlcnMgYXJlIAoKfCBDb2RlIHwgUGFyYW1ldGVyIE5hbWUgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfCBVbml0cyAgICB8CnwtLS0tLS18LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tfAp8IFBSQ1AgfCBQcmVjaXBpdGF0aW9uIHdydCAwMDAwLTAwMDAgTG9jYWwgVGltZSAgICAgICB8IOKFkiBvZiBtbSB8CnwgU05PVyB8IFNub3dmYWxsIHdydCAwMDAwLTAwMDAgTG9jYWwgVGltZSAgICAgICAgICAgIHwgbW0gICAgICAgfAp8IFNOV0QgfCBTbm93IGRlcHRoIHdydCAwMDAwLTAwMDAgTG9jYWwgVGltZSAgICAgICAgICB8IG1tICAgICAgIHwKfCBUTUFYIHwgTWF4aW11bSB0ZW1wZXJhdHVyZSB3cnQgMDAwMC0wMDAwIExvY2FsIFRpbWUgfCDihZIgb2YgwrBDIHwKfCBUTUlOIHwgTWluaW11bSB0ZW1wZXJhdHVyZSB3cnQgMDAwMC0wMDAwIExvY2FsIFRpbWUgfCDihZIgb2YgwrBDIHwKCkFuZCB5b3UgbWF5IGhhdmUgCgp8IENvZGUgfCBQYXJhbWV0ZXIgTmFtZSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8IFVuaXRzICAgIHwKfC0tLS0tLXwtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tfC0tLS0tLS0tLS18CnwgVEFWRyB8IEF2ZXJhZ2UgdGVtcGVyYXR1cmUgd3J0IDAwMDAtMDAwMCBVVEMgICAgICAgIHwg4oWSIG9mIMKwQyB8CgpUaGVzZSByZWFsbHkgYXJlIHRoZSBvbmx5IHBhcmFtZXRlcnMgSSBub3JtYWxseSBwdWxsLgoKCkknbSBkb2luZyB0aGlzIGJ5IGJydXRlIGZvcmNlIGFuZCBjaGFuZ2luZyB0aGUgdGVtcGVyYXR1cmUgdW5pdHMgdG8gwrBGIHdoaWxlIEkgYW0gYXQgaXQuLi4gIAoKYGBge3J9CgojIENoYW5naW5nIFVuaXRzCgoKCgpnaGNuX2RhdGEkdG1heCA9IGdoY25fZGF0YSR0bWF4LzEwICogOS4gLyA1LiArIDMyCmdoY25fZGF0YSR0bWluID0gZ2hjbl9kYXRhJHRtaW4vMTAgKiA5LiAvIDUuICsgMzIKCgpgYGAKCiMjIDQuMy4gUGF0Y2hpbmcgYW55IE1pc3NpbmcgVmFsdWVzCgpIZXJlLCB3ZSB3aWxsIGRvIGEgY2hlY2sgdG8gZW5zdXJlIGEgY29udGlub3VzIGRhdGFzZXQgdGhhdCBwbGFjZXMgIk5BIiBmb3IgbWlzc2luZyByZWNvcmRzIFtsZWZ0X2pvaW4oKV0oaHR0cHM6Ly93d3cucmRvY3VtZW50YXRpb24ub3JnL3BhY2thZ2VzL2Jhc2UvdmVyc2lvbnMvMy4wLjMvdG9waWNzL2NiaW5kKSBmdW5jdGlvbgoKYGBge3J9CgojIGNyZWF0ZSBhIGRhdGUgZnJhbWUgdG8gc2V0IHVzIHVwIHRvIGFjY29tb2RhdGUgbWlzc2luZyBkYXRhCgpkYWlseV90aW1lX2ZyYW1lID0gdGliYmxlKGRhdGUgPSBzZXEuRGF0ZShmcm9tID0gbWluKGdoY25fZGF0YSRkYXRlKSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdG8gICA9IG1heChnaGNuX2RhdGEkZGF0ZSksIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBieSAgID0gIjEgZGF5IikpCgoKZ2hjbl9kYXRhID0gbGVmdF9qb2luKGRhaWx5X3RpbWVfZnJhbWUsZ2hjbl9kYXRhLCBieT0iZGF0ZSIpCgpwcmludChnaGNuX2RhdGEpIAoKcmVtb3ZlKGRhaWx5X3RpbWVfZnJhbWUpCgpgYGAKCgpBbmQgd2l0aCB0aGF0IHdlIGNhbiBub3cgW2ZpbmFsbHldIHBsYXkhCgojIDUgTWFrZSBhIFNpbXBsZSBQbG90CgpMZXQncyBtYWtlIGEgc2ltcGxlIHBsb3QgZm9yIFRtaW4KCmBgYHtyfQoKZ2dwbG90KGRhdGEgPSBnaGNuX2RhdGEpICsgICAjIHVzZSBnaGNuX2RhdGEgYXMgdGhlIHNvdXJjZSBvZiB0aGUgZGF0YQogIAogIGFlcyh4ID0gZGF0ZSwgICAgICAgICAgICAgICMgc2VsZWN0IHNwZWNpZmljIGZpZWxkcyB0byBwbG90CiAgICAgIHkgPSB0bWluKSArCgogIHRoZW1lX2J3KCkgKyAgICAgICAgICAgICAgIyB1c2UgYSB2ZXJ5IHNpbXBsZSBwbG90aW5nIHRoZW1lCiAgCiAgZ2d0aXRsZShsYWJlbCAgICA9ICJEYWlseSBHbG9iYWwgSGlzdG9yaWNhbCBDbGltYXRlIERhdGEiLAogICAgICAgICAgc3VidGl0bGUgPSBzdHJfYyhuY2RjX2lkcyRuYW1lLAogICAgICAgICAgICAgICAgICAgICAgICAgICAiIDo6ICIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0YXRpb25pZF9mb3JfbmNkY3N0YXRpb25zKSkgKyAKICAKICB4bGFiKGxhYmVsID0gIkRhdGUiKSArCiAgCiAgeWxhYihsYWJlbCA9ICJNaW5pbXVtIERhaWx5IFRlbXBlcmF0dXJlICjCsEYpIikgKwogIAogIGdlb21fbGluZShjb2xvciA9ICJibHVlIikgICMgbWFrZSBhIHNpbXBsZSBsaW5lIHBsb3QgKGFuZCBtYWtlIGl0IHBpbmspCgoKYGBgCgpXZSBjYW4gYWxzb3MgdGFrZSBhIGNsb3NlIGxvb2sgYXQgYW55IGZsYWdzIGZvciB0aGF0IHBhcmFtZXRlci4KCgpgYGB7cn0KCiMgZ2V0IHRoZSB0bWluIHJlY29yZCBpbiB0aGlzIGNhc2UuCgpnaGNuX3RtaW4gPSBnaGNuX2RhdGEgJT4lIHNlbGVjdChjKCJkYXRlIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBlbmRzX3dpdGgoInRtaW4iKSkpCgpnaGNuX3RtaW4gJT4lIGZpbHRlcihxZmxhZ190bWluICE9ICIgIikKCmBgYApgYGB7cn0KCmdncGxvdChkYXRhID0gZ2hjbl9kYXRhKSArICAgIyB1c2UgZ2hjbl9kYXRhIGFzIHRoZSBzb3VyY2Ugb2YgdGhlIGRhdGEKICAKICBhZXMoeCA9IGRhdGUsICAgICAgICAgICAgICAjIHNlbGVjdCBzcGVjaWZpYyBmaWVsZHMgdG8gcGxvdAogICAgICB5ID0gdG1heCkgKwoKICB0aGVtZV9idygpICsgICAgICAgICAgICAgICMgdXNlIGEgdmVyeSBzaW1wbGUgcGxvdGluZyB0aGVtZQogIAogIGdndGl0bGUobGFiZWwgICAgPSAiRGFpbHkgR2xvYmFsIEhpc3RvcmljYWwgQ2xpbWF0ZSBEYXRhIiwKICAgICAgICAgIHN1YnRpdGxlID0gc3RyX2MobmNkY19pZHMkbmFtZSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgIiA6OiAiLAogICAgICAgICAgICAgICAgICAgICAgICAgICBzdGF0aW9uaWRfZm9yX25jZGNzdGF0aW9ucykpICsgCiAgCiAgeGxhYihsYWJlbCA9ICJEYXRlIikgKwogIAogIHlsYWIobGFiZWwgPSAiTWF4aW11bSBEYWlseSBUZW1wZXJhdHVyZSAowrBGKSIpICsKICAKICBnZW9tX2xpbmUoY29sb3IgPSAicmVkIikgICMgbWFrZSBhIHNpbXBsZSBsaW5lIHBsb3QgKGFuZCBtYWtlIGl0IHBpbmspCgoKYGBgCgpXZSBjYW4gYWxzb3MgdGFrZSBhIGNsb3NlIGxvb2sgYXQgYW55IGZsYWdzIGZvciB0aGF0IHBhcmFtZXRlci4KCgpgYGB7cn0KCiMgZ2V0IHRoZSB0bWluIHJlY29yZCBpbiB0aGlzIGNhc2UuCgpnaGNuX3RtYXggPSBnaGNuX2RhdGEgJT4lIHNlbGVjdChjKCJkYXRlIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBlbmRzX3dpdGgoInRtYXgiKSkpCgpnaGNuX3RtYXggJT4lIGZpbHRlcihxZmxhZ190bWF4ICE9ICIgIikKCmBgYAoKIyA2IE1ha2UgYSBGYW5jeSBQbG90IHRvIFNob3cgQ2hhbmdlcyBpbiBDbGltYXRlCgpOb3cgbGV0J3MgZmFuY3kuICBMZXQncyBzZWUgaG93IG1heCBhbmQgbWluIHRlbXBlcnR1cmVzIGZvciBlYWNoIG1vbnRoIGNoYW5nZXMgb3ZlciB0aW1lLgoKTGV0J3MgY2hvc2Ugc29tZSAzMC15IHBlcmlvZHMgYXQgdGhlIGJlZ2lubmluZyBhbmQgZW5kIG9mIHRoZSByZWNvcmQganVzdCBhcyBhIGRlbW8uICAoWW91IGNhbiBhZGQgbW9yZSBwZXJpb2RzIGlmIHlvdSB3YW50KQoKTGV0J3MgY2hvc2UgdHdvIHBhcmFtZXRlcnMuIE1heCBhbmQgTWluaW11bSBUZW1wZXJhdHVyZXMKCkZpcnN0LCBsZXQncyBjaGFuZ2UgdGhlIGRhdGEgZnJhbWUgc28gdGhhdCB3ZSBoYXZlIG9ubHkgbWluIGFuZCBtYXggdGVtcGVyYXR1cmVzCgpgYGB7cn0KCiMgcHVsbCBvbmx5IHRoZSBtaW4gYW5kIG1heCB0ZW1wcwoKdGVtcF9kYXRhID0gZ2hjbl9kYXRhICU+JSBzZWxlY3QoYyhkYXRlLCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0bWF4LCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0bWluKSkKCmBgYAoKCldlJ2xsIHN0YXJ0IGJ5IGNyZWF0aW5nIGFuIGFkZGl0aW9uYWwgZGF0YSBjb2x1bW4gdG8gcmVwcmVzZW50IHRoZSBkZWNhZGUgd2UnbGwgdXNlIHRoZSBbeWVhcigpXShodHRwczovL3d3dy5yZG9jdW1lbnRhdGlvbi5vcmcvcGFja2FnZXMvbHVicmlkYXRlL3ZlcnNpb25zLzEuNy40L3RvcGljcy95ZWFyKSBmdW5jdGlvbiB0byBkbyB0aGlzLgoKYGBge3J9CgojIENyZWF0ZSBhIHllYXIgcGFyYW1ldGVyCgp0ZW1wX2RhdGEkeWVhciA9IHllYXIodGVtcF9kYXRhJGRhdGUpIAoKIyBUdXJuIGl0IGludG8gYSBzdHJpbmcgdGhhdCBkaXNwbGF5cyB0aGUgc3RhcnQgYW5kIGVuZCB5ZWFyIG9mIHRoZSBkZWNhZGUKCmBgYAoKQW5kIGFkZCBhbm90aGVyIHZhcmlhYmxlIGZvciB0aGUgY2FsZW5kYXIgbW9udGgKCmBgYHtyfQoKIyBhZGQgbW9udGgKCnRlbXBfZGF0YSRtb250aCA9IG1vbnRoKHggICAgID0gdGVtcF9kYXRhJGRhdGUsCiAgICAgICAgICAgICAgICAgICAgICAgIGxhYmVsID0gVFJVRSwKICAgICAgICAgICAgICAgICAgICAgICAgYWJiciAgPSBUUlVFKQoKYGBgCgoKQW5kIGZpbHRlciBvbmx5IHRoZSBwZXJpb2RzIHdlIHdhbnQgdG8gdXNlCgpgYGB7cn0KCiMgZmlyc3QgZnVsbCB5ZWFyICh1c2UgdGhlIGxvd2VzdCBkYXRlIHdoZXJlIHRoZSBET1kgdmlhIHlkYXkgPT0gMSkKCnBlcmlvZF8xX3llYXJfc3RhcnQgPSB5ZWFyKG1pbih0ZW1wX2RhdGEkZGF0ZVt5ZGF5KHRlbXBfZGF0YSRkYXRlKT09MV0pKQpwZXJpb2RfMV95ZWFyX2VuZCAgID0gcGVyaW9kXzFfeWVhcl9zdGFydCArICgzMCAtIDEpCgpwZXJpb2RfMV9zdHJpbmcgICAgID0gc3RyX2MocGVyaW9kXzFfeWVhcl9zdGFydCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICItIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgIHBlcmlvZF8xX3llYXJfZW5kLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgc2VwID0gIiIpCgoKIyBtb3N0IHJlY2VudCB5ZWFyIHBlcmlvZAoKcGVyaW9kXzNfeWVhcl9zdGFydCA9IDIwMTggLSAoMzAgLSAxKQpwZXJpb2RfM195ZWFyX2VuZCAgID0gMjAxOApwZXJpb2RfM19zdHJpbmcgICAgID0gc3RyX2MocGVyaW9kXzNfeWVhcl9zdGFydCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICItIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgIHBlcmlvZF8zX3llYXJfZW5kLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgc2VwID0gIiIpICAKCiMgYSBwZXJpb2QgaW4gdGhlIG1pZGRsZQoKcGVyaW9kXzJfeWVhcl9zdGFydCA9IHJvdW5kKChwZXJpb2RfMV95ZWFyX3N0YXJ0ICsgcGVyaW9kXzNfeWVhcl9zdGFydCkgLyAyICkKcGVyaW9kXzJfeWVhcl9lbmQgICA9IHJvdW5kKChwZXJpb2RfMV95ZWFyX2VuZCAgICsgcGVyaW9kXzNfeWVhcl9lbmQpIC8gMiApCnBlcmlvZF8yX3N0cmluZyAgICAgPSBzdHJfYyhwZXJpb2RfMl95ZWFyX3N0YXJ0LAogICAgICAgICAgICAgICAgICAgICAgICAgICAgIi0iLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgcGVyaW9kXzJfeWVhcl9lbmQsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICBzZXAgPSAiIikgIAogIAoKIyBleHRyYWN0IHRoZXNlIHR3byBwZXJpb2RzCgp0ZW1wX2RhdGEgPSB0ZW1wX2RhdGEgJT4lIAogIGZpbHRlcigoICh5ZWFyKGRhdGUpID49IHBlcmlvZF8xX3llYXJfc3RhcnQpICAmIAogICAgICAgICAgICh5ZWFyKGRhdGUpIDw9IHBlcmlvZF8xX3llYXJfZW5kICAgKSApIHwKICAgICAgICAgKCAoeWVhcihkYXRlKSA+PSBwZXJpb2RfMl95ZWFyX3N0YXJ0KSAgJiAKICAgICAgICAgICAoeWVhcihkYXRlKSA8PSBwZXJpb2RfMl95ZWFyX2VuZCAgICkgKSB8CiAgICAgICAgICggKHllYXIoZGF0ZSkgPj0gcGVyaW9kXzNfeWVhcl9zdGFydCkgICYgCiAgICAgICAgICAgKHllYXIoZGF0ZSkgPD0gcGVyaW9kXzNfeWVhcl9lbmQgICApICkgKQoKdGVtcF9kYXRhID0gdGVtcF9kYXRhICU+JSAKICBtdXRhdGUoUGVyaW9kID0gY2FzZV93aGVuKCh5ZWFyKGRhdGUpID49IHBlcmlvZF8xX3llYXJfc3RhcnQpICAmICh5ZWFyKGRhdGUpIDw9IHBlcmlvZF8xX3llYXJfZW5kICkgfiBwZXJpb2RfMV9zdHJpbmcsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAoeWVhcihkYXRlKSA+PSBwZXJpb2RfMl95ZWFyX3N0YXJ0KSAgJiAoeWVhcihkYXRlKSA8PSBwZXJpb2RfMl95ZWFyX2VuZCApIH4gcGVyaW9kXzJfc3RyaW5nLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgKHllYXIoZGF0ZSkgPj0gcGVyaW9kXzNfeWVhcl9zdGFydCkgICYgKHllYXIoZGF0ZSkgPD0gcGVyaW9kXzNfeWVhcl9lbmQgKSB+IHBlcmlvZF8zX3N0cmluZykpCgp0ZW1wX2RhdGEkUGVyaW9kID0gYXMuZmFjdG9yKHRlbXBfZGF0YSRQZXJpb2QpCgpgYGAKCgpNYWtlIGEgcmlkZ2UgcGxvdCBieSBtb250aAoKCmBgYHtyfQoKIyByaWRnZXBsb3QKCmdncGxvdChkYXRhID0gdGVtcF9kYXRhKSArCiAgCiAgICAKICBhZXMoeCAgICA9IHRtYXgsCiAgICAgIHkgICAgPSBmY3RfcmV2KG1vbnRoKSwKICAgICAgZmlsbCA9IFBlcmlvZCkgKwogIAogIHRoZW1lX2J3KCkgKyAKICAKICAgIAogIGdndGl0bGUobGFiZWwgICAgPSAiRGFpbHkgR2xvYmFsIEhpc3RvcmljYWwgQ2xpbWF0ZSBEYXRhIiwKICAgICAgICAgIHN1YnRpdGxlID0gc3RyX2MobmNkY19pZHMkbmFtZSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgIiA6OiAiLAogICAgICAgICAgICAgICAgICAgICAgICAgICBzdGF0aW9uaWRfZm9yX25jZGNzdGF0aW9ucykpICsgCiAgCiAgICAKICB5bGFiKGxhYmVsID0gIk1vbnRoIikgKwogIAogIHhsYWIobGFiZWwgPSAiRGFpbHkgVGVtcGVyYXR1cmUgKMKwRikiKSArCiAgCiAgCgogIHNjYWxlX2ZpbGxfbWFudWFsKHZhbHVlcz1jKCJibHVlIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAieWVsbG93IiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAicmVkIikpKwoKICAKICBnZW9tX2RlbnNpdHlfcmlkZ2VzMihhbHBoYSA9IDAuMjUsCiAgICAgICAgICAgICAgICAgICAgICAgY29sb3IgPSBOQSkgKwoKICBnZW9tX2RlbnNpdHlfcmlkZ2VzMihtYXBwaW5nID0gYWVzKHggICAgPSB0bWluLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgeSAgICA9IGZjdF9yZXYobW9udGgpLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZmlsbCA9IFBlcmlvZCksCiAgICAgICAgICAgICAgICAgICAgICAgYWxwaGEgICA9IDAuMjUsCiAgICAgICAgICAgICAgICAgICAgICAgY29sb3IgICA9IE5BKSAKCmBgYAoKIyBTQ0FWRU5HRVIgSFVOVAoKMSkgVGFrZSB0aGUgZ2NobiBkYXRhIHNldAoyKSBGaW5kIFllYXJseSBNYXgtTWF4IFRlbXAsIE1pbi1NaW4gVGVtcCwgTWF4LURhaWx5IFByZWNpcAozKSBUcnkgeW91ciBtZXRob2QgZm9yIGdldHRpbmcgbWF4IG1pbiByZXR1cm5zIAoKYGBge3J9Cgp3cml0ZS5jc3YoeCAgICA9ICBnaGNuX2RhdGEsIAogICAgICAgICAgZmlsZSA9IHN0cl9jKCIuL0dIQ05fIiwKICAgICAgICAgICAgICAgICAgICAgICB1bmlxdWUoc3RhdGlvbmlkX2Zvcl9naGNuX3B1bGwpLAogICAgICAgICAgICAgICAgICAgICAgICIuY3N2IikpCmBgYAoK